Life In a World of Ubiquitous Sensing

Discovering the three dimensional structure of an object is important for a variety of robot tasks. Single sensor systems such as machine vision systems cannot reliably compute three dimensional structure in unconstrained environments. Active, exploratory tactile sensing can be used to complement passive stereo vision data to derive robust surface and feature descriptions of objects. The control for tactile sensing is provided by the vision system which provides regions of interest that the tactile system can explore. The descriptions of surfaces and features are accurate and can be used in a later matching phase against a model data base of objects to identify the object and its position and orientation in space. 1. INTRODUCTION Robotic systems are being built to perform complex tasks such as object recognition, grasping, manipulation and assembly. A common thread in all these tasks is the ability to understand the three dimensional geometric and topological structure of the objects involved. The first step in discovering the underlying three dimensional structure of an object through sensing is to compute depth and orientation at each point on the object, or what has been termed by Marr [IO] as a “2% D” sketch. Currently, there are several sensing systems that can derive depth from a scene. Among these are laser range finders [17,9, I], photometric stereo [7] and binocular stereo [3]. Determining which sensor to use is chiefly determined by the task domain. Laser imaging is potentially hazardous and has difficulty with shiny metal reflective surfaces. At present, it is a more expensive depth sensing technology than the other methods mentioned above. Photometric stereo puts great demands on the illumination in the scene and on properly understanding the reflectance properties of the objects to be viewed. In choosing a system to sense depth in general, unconstrained environments, binocular stereo has the advantage of low cost and ability to perform over a wide range of illuminations and object domains. It is also a well understood and simple ranging method, which motivates its use in a generalized robotics environment where many different task and object domains may be in effect. However, used as a single robotics sensing system, stereo has clear deficiencies. If there is a lack of detail on the object, only sparse depth measurements are possible. If too much detail is present, the matching process between image vents can easily become confused. Detail also causes a marked degradation in performance as the ~H2282-2/86/0000/0126~01.00 1986 IEEE potential match space increases. To overcome the limitations of such a sensor system a useful approach is to use multiple sensors. Multiple sensors can be used in a complementary fashion to extract more information from an environment than a single sensor [16,12]. Tactile sensing is a good choice for a complementary sensor to vision in a generalized robotics environment for a number of reasons. Touch is able to directly measure the properties of objects we desire: their position and surface orientation. It also can sense visually occluded areas of a scene, making it more useful than the other ranging devices mentioned. Lastly, it is a low cost sensor that is required for tasks such as grasping and manipulation, making it a necessary part of a robotic system. A system using passive stereo vision and active exploratory tactile sensing to recognize common kitchen items such as mugs, bowls, pitchers and plates has been built and is described in detail in [2]. A key component of this system are modules that create surface and feature descriptions of objects. This paper describes the procedures used to generate the surface and feature descriptions and reports results achieved with the method for a number of real objects. The experimental hardware is shown in figure 1. The objects to be recognized are rigidly placed on the worktable and imaged by a pair of CCD cameras. The tactile sensor is mounted on a 6 degree of freedom PUMA manipulator that receives feedback from the tactile sensor allowing it to move across the surfaces of objects reporting contacts. The sections below describe how the vision and touch are integrated into robust surface and Figure 1. Experimental hardware. I26 Stereo cameras Tac@le sensor Force Cup Bolted to the table Grasp Lab, ca. 1985